Method and apparatus for generating a clinician quality assurance scoreboard

ABSTRACT

The present invention provides a quality assurance system and method that generates a quality assurance (QA) scorecard for clinicians that participate in a radiological-based medical imaging study using digital imaging technologies. According to one embodiment, client computers, servers, imaging devices, databases, and/or other components may be coupled to provided a unified data collection system. According to one embodiment, systems and methods are provided that analyze various parameters that are derived from the unified data collection system to calculate a QA score for the clinician. The QA score provides a combined subjective and objective feedback system that includes performance evaluations from other users, including radiologists, technologists and patients. According to one embodiment, the feedback may be provided in real-time.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from U.S. Provisional PatentApplication No. 60/762,859, dated Jan. 30, 2006, U.S. Provisional PatentApplication No. 60/763,353, dated Jan. 31, 2006, U.S. Provisional PatentApplication No. 60/763,357, dated Jan. 31, 2006, U.S. Provisional PatentApplication No. 60/771,482, dated Feb. 9, 2006, U.S. Provisional PatentApplication No. 60/771,484, dated Feb. 9, 2006, the contents of whichare herein incorporated by reference in their entirety.

This application is related to the following concurrently filed commonlyowned U.S. Patent Applications entitled, “Method And Apparatus ForGenerating A Technologist Quality Assurance Scorecard” (Attorney DocketNos. 71486.0036 filed Jan. 30, 2007); “Method And Apparatus ForGenerating A Patient Quality Assurance Scorecard” (Attorney Docket Nos.71486.0037 filed Jan. 30, 2007); “Method And Apparatus For Generating AnAdministrative Quality Assurance Scorecard” (Attorney Docket Nos.71486.0038 filed Jan. 30, 2007); and “Method And Apparatus ForGenerating A Radiologist Quality Assurance Scorecard” (Attorney DocketNos. 71486.0039 filed Jan. 30, 2007), the contents of all of which areherein incorporated by reference in their entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a quality assurance (QA) system andmethod that quantitatively rates users that perform and/or participatein medical procedures, particularly in the area of radiology. Thepresent invention relates to systems, methods and computer-basedsoftware programs that analyze data and generate QA scorecards forclinicians. In the process of doing so, a number of objective data arecollected for real-time and future analysis, thereby providing objectivefeedback to clinicians for continuing quality improvement. In the end,the invention is intended to improve patient safety and overall clinicaloutcomes.

2. Description of the Related Art

The first and foremost priority for any QA program is to improve qualityof service. As QA programs are implemented in the medical field, theultimate goal is to improve patient care. To accomplish this goal,products and/or services should offer procedures for increasingaccountability and improving feedback among users that participate in amedical study. This ultimately will enhance patient diagnosis and/ortreatment, which leads to objective improvements in overall healthoutcomes.

Medical imaging has undergone a transition from film-based imagingtechnologies to digital imaging technologies. Digital imagingtechnologies provide digital processing capabilities, such as imagecapture, image archive, image transfer, and image display that may beshared among users to the medical study. Digital imaging technologiesfurther allow data that is associated with the digital processingoperations to be captured and combined with the underlying digitalimaging processing operations.

Accordingly, a need exists to leverage digital imaging technologies toincrease accountability and improve feedback among users thatparticipate in a medical study.

SUMMARY OF THE INVENTION

The present invention relates to systems, methods and computer-basedsoftware programs that provide a QA scorecard for users that participatein a radiology imaging study. The QA Scorecard provides the frameworkfor developing a comprehensive medical imaging QA program that definesobjective benchmarks. One of ordinary skill in the art will readilyrecognize that this invention may be applied to other medicaldisciplines, as well as to non-medical disciplines.

According to one embodiment, the invention is directed toradiological-based medical studies using digital imaging technologies.The medical studies are performed by users that perform discrete tasksin an imaging study workflow sequence. According to one embodiment ofthe invention, users may include clinicians, radiologists,technologists, administrators and patients, among other users. A typicalworkflow sequence includes imaging exam ordering, imaging examscheduling, imaging exam acquisition, imaging exam processing, imagingexam archiving, imaging exam distribution, imaging exam display, imagingexam navigation, imaging exam interpretation, imaging exam reporting,communication and billing, among other sequences.

According to one embodiment of the invention, client computers, one ormore servers, the imaging devices, one or more databases, and/or othercomponents may be coupled via a wired media, a wireless media, or acombination of the foregoing to provided a unified data collectionsystem.

According to one embodiment of the invention, the client computers mayinclude any number of different types of client terminal devices, suchas personal computers, laptops, smart terminals, personal digitalassistants (PDAs), cell phones, portable processing devices that combinethe functionality of one or more of the foregoing or other clientterminal devices.

According to one embodiment, the client computer may include clientcomputer agent modules that gather client computer monitoring data basedon user actions that are performed. According to another embodiment ofthe invention, user action data may include accessing digital images,reviewing digital images, manipulating digital images, marking digitalimages, storing digital images, forwarding digital images, adjustingexposure parameters on digital imaging devices, generating a report,generating a textual report, dictating a report, entering information,conducting continuing medical education (CME) triggered by performingthe medical examination, and/or performing other user actions.

According to one embodiment, the client computer may include clientcomputer agent modules that gather client computer monitoring data basedon computer actions that are performed. According to one embodiment ofthe invention, the client computer agent modules also may gather clientcomputer specification data, such as IP address data, processing speeddata, and other client computer specification data. According to oneembodiment of the invention, the client monitoring data and/or clientcomputer specification data may be provided in real-time. According toanother embodiment of the invention, the client monitoring data and/orclient computer specification data may be employed to calculate user QAmetrics.

According to one embodiment of the invention, the imaging devices mayinclude any number of different types of imaging devices, such asmagnetic resonance imaging (MRI) devices, computer tomograph (CT)imaging devices, angiograph imaging device, ultrasound imaging devicesor other imaging devices.

According to one embodiment of the invention, the imaging devices mayinclude, or be modified to include, imaging device agent modules. Theimaging device agent modules may operate to provide data gathering anddata exchange functionality. According to one embodiment, the inventionmay enable monitoring of actions that are performed on the imagingdevices.

According to one embodiment of the invention, the imaging device agentmodules may associate imaging device identifying information withactions that are performed on the imaging devices. According to oneembodiment of the invention, data monitoring features may be employed togenerate imaging device audit logs. According to one embodiment of theinvention, image device audit logs may be produced to reconstructactions, such as user actions, imaging device actions, and other actionsthat are performed on (or by) the imaging devices.

According to one embodiment of the invention, databases or informationsources include a Hospital Information System (HIS) 10, a RadiologyInformation System (RIS) 20, a Picture Archiving and CommunicationSystem (PACS) 30, an Electronic Medical Record (EMR), a patient specificimaging datasheet and/or other information sources.

According to one embodiment of the invention, the server may include amerging module that receives data from all devices that are networked tothe server, including the client computers, the imaging devices, and/ordatabases or information sources. According to one embodiment of theinvention, the received data may include at least client computer auditlog data and/or image device audit log data. According to oneembodiment, the merging module merges data that is captured during amedical examination, including user action data, client computer actiondata, imaging device action data, and other data.

According to one embodiment, a quantifiable list of pre-defined clinicalperformance parameters may be used by the program to measure overallperformance of the clinician, or practicing physician, such as theutilization and medical imaging services that are provided in a clinicalpractice, among other pre-defined parameters.

According to one embodiment of the invention, clinical performancemetrics may be calculated by the program based on predefined parameters,including completeness of data input, such as clinical history,laboratory data, physical exam findings; exam appropriateness, such asusing defined appropriateness criteria; utilization patterns, includingeconomic outcomes, clinical outcomes, and/or medico-legal outcomes; apatient safety profile, such as requested use of ionizing radiation,contrast, invasive procedures; communication/reporting, including theavailability of imaging data, the receipt of imaging data, and/orradiologist consultations; timeliness, including time to initiateclinical action; feedback provided to the patient and specialists, suchas the radiologist; participation in data collection and analysis,including outcomes analysis, reporting, and/or diagnostic accuracy;education and training, including imaging services and new technologies;peer review, including discretionary assessment of clinical performanceas it relates to imaging services and patient diagnosis/treatment, amongother predefined parameters.

According to one embodiment of the invention, the data that is collectedduring the imaging study may analyzed by a metrics module that performsprospective and retrospective trending analysis. The prospective andretrospective trending analysis enables automatic detection of immediateand recurrent problems, as they relate to equipment, personnel, datainput, and overall workflow. The result of this automated technical QAanalysis is that an automated and normalized analysis may be performedthat minimizes subjectivity and human bias, among providing otherbenefits.

According to one embodiment of the invention, the metrics module mayautomatically tally and record QA scores. The QA scores may becross-referenced by the computer program to a number of independentvariables including a technologist identifier, imaging modality, examtype, patient demographics, patient characteristics, patient bodyhabitus, exposure parameters, image processing, exam location,equipment, day/time of exam for trending analysis, radiologistidentification, referring clinician, clinical indication, among othervariables.

According to one embodiment, the metrics module may analyze data that isassociated with a defined list of quality assurance (QA) benchmarks toobjectively evaluate clinicians, quantify a relative success of servicedelivery and provide educational (data-driven) feedback in order tooptimize clinical performance, among other benefits. The QA metrics maybe tied to economic incentives, such as a pay for performance (P4P)systems, to create financial rewards for those practitioners thatprovide high levels of quality-oriented service deliverables.

According to one embodiment of the invention, a standard tag may becreated by the program within the various informational sources toidentify individual QA data parameters. The communication module mayextract the parameters from the CPOE entries to calculate metrics andgenerate a QA score for the clinician.

According to one embodiment of the invention, the QA metrics module mayanalyze various parameters to calculate a QA score for the clinician.According to one embodiment, the time-stamped data is a part ofobjective data analysis. Imaging departments may utilize a program torecord individual time-stamped data throughout the course of the imagingcycle, from the time an imaging exam is electronically ordered to thetime the imaging report issued and reviewed. After the image report isreceived, time-stamped data may be tracked by the program within theEMR, which records clinician actions, in the form of recording progressnotes, consultations, and the ordering of clinical tests, imagingstudies, and various treatment options (e.g. drug therapy). In eithercase, the QA scorecard program enables the clinician to enter dataelectronically into the EMR. This is time-stamped data may be recordedinto a QA database for subsequent analysis. One such analysis mayinclude an assessment of the time incurred between the imaging exam andinitiation of clinical treatment.

According to one embodiment, in order to optimize safety concerns andrecord/track cumulative data, the QA scorecard program provides patientsafety data at any location where the patient is seeking and/orreceiving medical imaging services. By storing the QA Scorecard datawithin a universal EMR, this data is made accessible to appropriatehealthcare providers at any location.

According to one embodiment, the QA scorecard program may track, recordand analyze longitudinal patient-specific safety data andclinician-specific safety data, both an individual patient and groupbasis. This provides insight as to whether individual clinicians areover-utilizing certain types of “higher risk” imaging studies andprovides educational feedback to specific clinicians. Additionally,mandatory educational resources may be forwarded to targeted cliniciansfor completion before imaging privileges are re-instated. This“clinician safety profile” data and trending analyses may be correlatedby the program with local, regional, and national norms, with dataavailable to third party payers and insurers to assist with economicincentive programs (P4P) to encourage improved performance andcontinuing medical education, as it relates to medical imaging safetyfactors.

According to one embodiment, a combined subjective and objectivefeedback system, method and computer program are provided that supplydata to clinicians as to how their performance is perceived by imagingservice providers, such as radiologists, and customers, such aspatients. According to one embodiment, the feedback may be provided inreal-time.

Thus has been outlined, some features consistent with the presentinvention in order that the detailed description thereof that followsmay be better understood, and in order that the present contribution tothe art may be better appreciated. There are, of course, additionalfeatures consistent with the present invention that will be describedbelow and which will form the subject matter of the claims appendedhereto.

In this respect, before explaining at least one embodiment consistentwith the present invention in detail, it is to be understood that theinvention is both limited in its application to the details ofconstruction and to the arrangements of the components set forth in thefollowing description or illustrated in the drawings. Methods andapparatuses consistent with the present invention are capable of otherembodiments and of being practiced and carried out in various ways.Also, it is to be understood that the phraseology and terminologyemployed herein, as well as the abstract included below, are for thepurpose of description and should not be regarded as limiting.

As such, those skilled in the art will appreciate that the conceptionupon which this disclosure is based may readily be utilized as a basisfor the designing of other structures, methods and systems for carryingout the several purposes of the present invention. It is important,therefore, that the claims be regarded as including such equivalentconstructions insofar as they do not depart from the spirit and scope ofthe methods and apparatuses consistent with the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a schematic diagram of a quality assurance scorecardsystem for radiology, according to one embodiment of the presentinvention.

FIG. 2 illustrates a schematic diagram of a quality assurance scorecardsystem for radiology, according to another embodiment of the presentinvention.

FIGS. 3A and 3B illustrate a flow chart of a workflow sequence qualityassurance program for image ordering from the perspective of aclinician, according to one embodiment of the present invention.

FIG. 4 illustrates a flow chart of a workflow sequence quality assuranceprogram for communication and reporting from the perspective of aclinician, according to one embodiment consistent with the presentinvention.

DESCRIPTION OF THE INVENTION

The present invention relates to systems, methods, and computer-basedsoftware programs for generating quality assurance (QA) metrics, orscorecards, for clinicians that participate in radiological-basedmedical studies.

Radiological-based medical studies of the present invention areconducted using digital imaging technologies. The medical studies areperformed by many users that perform discrete tasks in an imaging studyworkflow sequence. Typically, the workflow sequence is initiated by aclinician, such as a family practice physician, that examines a patientand orders an imaging examination. The clinician's staff contacts animaging center and schedules the imaging examination. At the imagingcenter, a technologist operates one or more imaging devices to acquirepatient images. In some cases, the number of patient images taken maytotal several hundred or several thousand images. During the imageacquisition operation, the technologist may process the images,including applying algorithms to the raw imaging data in order toenhance selected image features, reconstructing the raw image data indifferent ways to optimize imaging views, and/or performing other imageprocessing. Upon completion of the imaging examination, the patient maybe discharged from the imaging facility and the images may be locallystored. Generally, imaging administrators periodically obtain the imagesfrom the local storage devices and archive the images in a database,such as a Picture Archival Retrieval System (PACS) and/or other imagingdatabases. The images may be archived and retrieved based on selectedcriteria, including patient name, patient reference number, patientidentifier, physician identifier, and/or other selected criteria.

After the images are archived, the images may be distributed to one ormore specialists, such as a radiologist. Alternatively, a message may becommunicated to one or more specialists advising the specialists thatthe images are available and providing instructions for accessing thearchived images from the PACS or other imaging databases. Theradiologist may access the PACS or other imaging databases and mayperform image display and image navigation functions. The radiologistinterprets the images and may access decision support tools or otherinterpretation tools during the image interpretation process. Followingthe image interpretation, the radiologist may generate a report and/orotherwise communicate the image study results to the referringclinician, among others. Upon completion of the imaging process, theradiologist, an administrator, and/or other service provider may performbilling operations. Additionally, an administrator may be tasked withdefining the lines of responsibility for the participants of the imagingexam and for developing a comprehensive program that ensures appropriatelevels of quality, while balancing economics, service deliverables andproductivity. One of ordinary skill in the art will readily appreciatethat the imaging study workflow sequence may include other operations.

According to one embodiment of the invention illustrated in FIG. 1,medical (radiological) applications may be implemented using the QAscorecard system 100. The QA scorecard system 100 is designed tointerface with existing information systems such as a HospitalInformation System (HIS) 10, a Radiology Information System (RIS) 20, aradiographic device 21, and/or other information systems that may accessa computed radiography (CR) cassette or direct radiography (DR) system,a CR/DR plate reader 22, a Picture Archiving and Communication System(PACS) 30, and/or other systems. The QA scorecard system 100 may bedesigned to conform with the relevant standards, such as the DigitalImaging and Communications in Medicine (DICOM) standard, DICOMStructured Reporting (SR) standard, and/or the Radiological Society ofNorth America's Integrating the Healthcare Enterprise (IHE) initiative,among other standards.

According to one embodiment, bi-directional communication between the QAscorecard system 100 of the present invention and the informationsystems, such as the HIS 10, RIS 20, radiographic device 21, CR/DR platereader 22, and PACS 30, etc., may be enabled to allow the QA scorecardsystem 100 to retrieve and/or provide information from/to these systems.According to one embodiment of the invention, bi-directionalcommunication between the QA scorecard system 100 of the presentinvention and the information systems allows the QA scorecard system 100to update information that is stored on the information systems.According to one embodiment of the invention, bi-directionalcommunication between the QA scorecard system 100 of the presentinvention and the information systems allows the QA scorecard system 100to generate desired reports and/or other information.

The QA scorecard system 100 of the present invention includes a clientcomputer 101, such as a personal computer (PC), which may or may not beinterfaced or integrated with the PACS 30. The client computer 101 mayinclude an imaging display device 102 that is capable of providing highresolution digital images in 2-D or 3-D, for example. According to oneembodiment of the invention, the client computer 101 may be a mobileterminal if the image resolution is sufficiently high. Mobile terminalsmay include mobile computing devices, a mobile data organizer (PDA), orother mobile terminals that are operated by the user accessing theprogram 110 remotely.

According to one embodiment of the invention, an input device 104 orother selection device, may be provided to select hot clickable icons,selection buttons, and/or other selectors that may be displayed in auser interface using a menu, a dialog box, a roll-down window, or otheruser interface. The user interface may be displayed on the clientcomputer 101. According to one embodiment of the invention, users mayinput commands to a user interface through a programmable stylus,keyboard, mouse, speech processing device, laser pointer, touch screen,or other input device 104.

According to one embodiment of the invention, the input or otherselection device 104 may be implemented by a dedicated piece of hardwareor its functions may be executed by code instructions that are executedon the client processor 106. For example, the input or other selectiondevice 104 may be implemented using the imaging display device 102 todisplay the selection window with a stylus or keyboard for entering aselection.

According to another embodiment of the invention, symbols and/or iconsmay be entered and/or selected using an input device 104, such as amulti-functional programmable stylus. The multi-functional programmablestylus may be used to draw symbols onto the image and may be used toaccomplish other tasks that are intrinsic to the image display,navigation, interpretation, and reporting processes, as described inU.S. patent application Ser. No. 11/512,199 filed on Aug. 30, 2006, theentire contents of which are hereby incorporated by reference. Themulti-functional programmable stylus may provide superior functionalitycompared to traditional computer keyboard or mouse input devices.According to one embodiment of the invention, the multi-functionalprogrammable stylus also may provide superior functionality within thePACS and Electronic Medical Report (EMR).

According to one embodiment of the invention, the client computer 101may include a processor 106 that provides client data processing.According to one embodiment of the invention, the processor 106 mayinclude a central processing unit (CPU) 107, a parallel processor, aninput/output (I/O) interface 108, a memory 109 with a program 110 havinga data structure 111, and/or other components. According to oneembodiment of the invention, the components all may be connected by abus 112. Further, the client computer 101 may include the input device104, the image display device 102, and one or more secondary storagedevices 113. According to one embodiment of the invention, the bus 112may be internal to the client computer 101 and may include an adapterthat enables interfacing with a keyboard or other input device 104.Alternatively, the bus 112 may be located external to the clientcomputer 101.

According to one embodiment of the invention, the image display device102 may be a high resolution touch screen computer monitor. According toone embodiment of the invention, the image display device 102 mayclearly, easily and accurately display images, such as x-rays, and/orother images. Alternatively, the image display device 102 may beimplemented using other touch sensitive devices including tabletpersonal computers, pocket personal computers, plasma screens, amongother touch sensitive devices. The touch sensitive devices may include apressure sensitive screen that is responsive to input from the inputdevice 104, such as a stylus, that may be used to write/draw directlyonto the image display device 102.

According to another embodiment of the invention, high resolutiongoggles may be used as a graphical display to provide end users with theability to review images. According to another embodiment of theinvention, the high resolution goggles may provide graphical displaywithout imposing physical constraints of an external computer.

According to another embodiment, the invention may be implemented by anapplication that resides on the client computer 101, wherein the clientapplication may be written to run on existing computer operatingsystems. Users may interact with the application through a graphicaluser interface. The client application may be ported to other personalcomputer (PC) software, personal digital assistants (PDAs), cell phones,and/or any other digital device that includes a graphical user interfaceand appropriate storage capability.

According to one embodiment of the invention, the processor 106 may beinternal or external to the client computer 101. According to oneembodiment of the invention, the processor 106 may execute a program 110that is configured to perform predetermined operations. According to oneembodiment of the invention, the processor 106 may access the memory 109in which may be stored at least one sequence of code instructions thatmay include the program 110 and the data structure 111 for performingpredetermined operations. The memory 109 and the program 110 may belocated within the client computer 101 or external thereto.

While the system of the present invention may be described as performingcertain functions, one of ordinary skill in the art will readilyunderstand that the program 110 may perform the function rather than theentity of the system itself.

According to one embodiment of the invention, the program 110 that runsthe QA scorecard system 100 may include separate programs 110 havingcode that performs desired operations. According to one embodiment ofthe invention, the program 110 that runs the QA scorecard system 100 mayinclude a plurality of modules that perform sub-operations of anoperation, or may be part of a single module of a larger program 110that provides the operation.

According to one embodiment of the invention, the processor 106 may beadapted to access and/or execute a plurality of programs 110 thatcorrespond to a plurality of operations. Operations rendered by theprogram 110 may include, for example, supporting the user interface,providing communication capabilities, performing data mining functions,performing e-mail operations, and/or performing other operations.

According to one embodiment of the invention, the data structure 111 mayinclude a plurality of entries. According to one embodiment of theinvention, each entry may include at least a first storage area, orheader, that stores the databases or libraries of the image files, forexample.

According to one embodiment of the invention, the storage device 113 maystore at least one data file, such as image files, text files, datafiles, audio files, video files, among other file types. According toone embodiment of the invention, the data storage device 113 may includea database, such as a centralized database and/or a distributed databasethat are connected via a network. According to one embodiment of theinvention, the databases may be computer searchable databases. Accordingto one embodiment of the invention, the databases may be relationaldatabases. The data storage device 113 may be coupled to the server 120and/or the client computer 101, either directly or indirectly through acommunication network, such as a LAN, WAN, and/or other networks. Thedata storage device 113 may be an internal storage device. According toone embodiment of the invention, QA scorecard system 100 may include anexternal storage device 114. According to one embodiment of theinvention, data may be received via a network and directly processed.

According to one embodiment of the invention, the client computer 101may be coupled to other client computers 101 or servers 120. Accordingto one embodiment of the invention, the client computer 101 may accessadministration systems, billing systems and/or other systems, via acommunication link 116. According to one embodiment of the invention,the communication link 116 may include a wired and/or wirelesscommunication link, a switched circuit communication link, or mayinclude a network of data processing devices such as a LAN, WAN, theInternet, or combinations thereof. According to one embodiment of theinvention, the communication link 116 may couple e-mail systems, faxsystems, telephone systems, wireless communications systems such aspagers and cell phones, wireless PDA's and other communication systems.

According to one embodiment of the invention, the communication link 116may be an adapter unit that is capable of executing variouscommunication protocols in order to establish and maintain communicationwith the server 120, for example. According to one embodiment of theinvention, the communication link 116 may be implemented using aspecialized piece of hardware or may be implemented using a general CPUthat executes instructions from program 110. According to one embodimentof the invention, the communication link 116 may be at least partiallyincluded in the processor 106 that executes instructions from program110.

According to one embodiment of the invention, if the server 120 isprovided in a centralized environment, the server 120 may include aprocessor 121 having a CPU 122 or parallel processor, which may be aserver data processing device and an I/O interface 123. Alternatively, adistributed CPU 122 may be provided that includes a plurality ofindividual processors 121, which may be located on one or more machines.According to one embodiment of the invention, the processor 121 may be ageneral data processing unit and may include a data processing unit withlarge resources (i.e., high processing capabilities and a large memoryfor storing large amounts of data).

According to one embodiment of the invention, the server 120 also mayinclude a memory 124 having a program 125 that includes a data structure126, wherein the memory 124 and the associated components all may beconnected through bus 127. If the server 120 is implemented by adistributed system, the bus 127 or similar connection line may beimplemented using external connections. The server processor 121 mayhave access to a storage device 128 for storing preferably large numbersof programs 110 for providing various operations to the users.

According to one embodiment of the invention, the data structure 126 mayinclude a plurality of entries, wherein the entries include at least afirst storage area that stores image files. Alternatively, the datastructure 126 may include entries that are associated with other storedinformation as one of ordinary skill in the art would appreciate.

According to one embodiment of the invention, the server 120 may includea single unit or may include a distributed system having a plurality ofservers 120 or data processing units. The server(s) 120 may be shared bymultiple users in direct or indirect connection to each other. Theserver(s) 120 may be coupled to a communication link 129 that ispreferably adapted to communicate with a plurality of client computers101.

According to one embodiment, the present invention may be implementedusing software applications that reside in a client and/or serverenvironment. According to another embodiment, the present invention maybe implemented using software applications that reside in a distributedsystem over a computerized network and across a number of clientcomputer systems. Thus, in the present invention, a particular operationmay be performed either at the client computer 101, the server 120, orboth.

According to one embodiment of the invention, in a client-serverenvironment, at least one client and at least one server are eachcoupled to a network 220, such as a Local Area Network (LAN), Wide AreaNetwork (WAN), and/or the Internet, over a communication link 116, 129.Further, even though the systems corresponding to the HIS 10, the RIS20, the radiographic device 21, the CR/DR reader 22, and the PACS 30 (ifseparate) are shown as directly coupled to the client computer 101, itis known that these systems may be indirectly coupled to the client overa LAN, WAN, the Internet, and/or other network via communication links.According to one embodiment of the invention, users may access thevarious information sources through secure and/or non-secure internetconnectivity. Thus, operations consistent with the present invention maybe carried out at the client computer 101, at the server 120, or both.The server 120, if used, may be accessible by the client computer 101over the Internet, for example, using a browser application or otherinterface.

According to one embodiment of the invention, the client computer 101may enable communications via a wireless service connection. The server120 may include communications with network/security features, via awireless server, which connects to, for example, voice recognition.According to one embodiment, user interfaces may be provided thatsupport several interfaces including display screens, voice recognitionsystems, speakers, microphones, input buttons, and/or other interfaces.According to one embodiment of the invention, select functions may beimplemented through the client computer 101 by positioning the inputdevice 104 over selected icons. According to another embodiment of theinvention, select functions may be implemented through the clientcomputer 101 using a voice recognition system to enable hands-freeoperation. One of ordinary skill in the art will recognize that otheruser interfaces may be provided.

According to another embodiment of the invention, the client computer101 may be a basic system and the server 120 may include all of thecomponents that are necessary to support the software platform. Further,the present client-server system may be arranged such that the clientcomputer 101 may operate independently of the server 120, but the server120 may be optionally connected. In the former situation, additionalmodules may be connected to the client computer 101. In anotherembodiment consistent with the present invention, the client computer101 and server 120 may be disposed in one system, rather being separatedinto two systems.

Although the above physical architecture has been described asclient-side or server-side components, one of ordinary skill in the artwill appreciate that the components of the physical architecture may belocated in either client or server, or in a distributed environment.

Further, although the above-described features and processing operationsmay be realized by dedicated hardware, or may be realized as programshaving code instructions that are executed on data processing units, itis further possible that parts of the above sequence of operations maybe carried out in hardware, whereas other of the above processingoperations may be carried out using software.

The underlying technology allows for replication to various other sites.Each new site may maintain communication with its neighbors so that inthe event of a catastrophic failure, one or more servers 120 maycontinue to keep the applications running, and allow the system toload-balance the application geographically as required.

Further, although aspects of one implementation of the invention aredescribed as being stored in memory, one of ordinary skill in the artwill appreciate that all or part of the invention may be stored on orread from other computer-readable media, such as secondary storagedevices, like hard disks, floppy disks, CD-ROM, a carrier wave receivedfrom a network such as the Internet, or other forms of ROM or RAM eithercurrently known or later developed. Further, although specificcomponents of the system have been described, one skilled in the artwill appreciate that the system suitable for use with the methods andsystems of the present invention may contain additional or differentcomponents.

FIG. 2 illustrates the QA scorecard system 100 for providing QAassessments of clinicians that access a radiology system, according toone embodiment of the invention. According to one embodiment, the clientcomputers 101 a-101 n (hereinafter client computers 101), one or moreservers 120, the imaging devices 210 a-210 n (hereinafter imagingdevices 210), one or more databases (HIS 10, RIS 20, PACS 30, etc.),and/or other components may be coupled via a wired media, a wirelessmedia, or a combination of the foregoing. According to one embodiment ofthe invention, the client computers 101, the server 120, the imagingdevices 210, and the databases may reside in one or more networks, suchas an internet, an intranet, or a combination thereof.

According to one embodiment of the invention, the client computers 101may include any number of different types of client terminal devices,such as personal computers, laptops, smart terminals, personal digitalassistants (PDAs), cell phones, portable processing devices that combinethe functionality of one or more of the foregoing or other clientterminal devices.

According to another embodiment of the invention, the client computers101 may include several components, including processors, RAM, a USBinterface, a telephone interface, microphones, speakers, a stylus, acomputer mouse, a wide area network interface, local area networkinterfaces, hard disk drives, wireless communication interfaces, DVD/CDreaders/burners, a keyboard, a flat touch-screen display, a computerdisplay, and/or other components. According to yet another embodiment ofthe invention, client computers 101 may include, or be modified toinclude, software that may operate to provide data gathering and dataexchange functionality.

According to one embodiment of the invention, the client computers 101,the servers 120, and/or the imaging devices 210 may include severalmodules. The modular construction facilitates adding, deleting, updatingand/or amending modules therein and/or features within modules. Theclient computer 101 may include various modules, including a userinterface module 220, an authentication module 222, a communicationsmodule 224, an agent module 226, and/or other modules. The servers 120may include various modules, including a server communication module230, a merging module 231, a metrics module 232, a server authenticationmodule 234, a notification module 236, a scheduling module 244 a reportgenerating module 238, a sorting module 240, a billing module 242,and/or other modules. The imaging devices 210 may include variousmodules, including a communications module 212, an authentication module214, an agent module 216 and/or other modules, along with a localstorage device 219. It should be readily understood that a greater orlesser number of modules might be used. One skilled in the art willreadily appreciate that the invention may be implemented usingindividual modules, a single module that incorporates the features oftwo or more separately described modules, individual software programs,and/or a single software program.

According to one embodiment of the invention, the client computer 101may communicate through a networking application. According to anotherembodiment, the user interface modules 220 a-220 n (hereinafter userinterface modules 220) may support several interfaces including displayscreens, voice recognition systems, speakers, microphones, inputbuttons, and/or other interfaces. According to one embodiment of theinvention, the user interface modules 220 may display the application ona user interface associated with the client computer 101. According toone embodiment of the invention, select functions may be implementedthrough the client computer 101 by positioning an indicator overselected icons and manipulating an input device 104, such as a stylus, amouse, a keyboard, or other input devices.

With regard to user authentication, the authentication modules 222 a-222n (hereinafter user authentication modules 222) may employ one ofseveral different authentication schemes, as would be appreciated bythose skilled in the art. According to one embodiment of the invention,the user authentication modules 222 may prompt users to inputalphanumeric code or other identifying information. According to anotherembodiment of the invention, the user authentication modules 222 mayprompt users to provide biometric information (i.e., a thumbprintthrough a fingerprint scanner) or other suitable identifyinginformation. If the user is not identified, then the user may be invitedto resubmit the requested identification information or to take otheraction.

According to one embodiment of the invention, the client computers 101may include communication modules 224 a-224 n (hereinafter communicationmodules 224) for enabling the client computers 101 to communicate withsystems, including other client computers, the servers 120, the imagingdevices 210, the databases and/or other systems. The client computers101 may communicate via communications media 201 such as, for example,any wired and/or wireless media. Communications between the clientcomputers 101, the imaging devices 210, the servers 120, and thedatabases may occur substantially in real-time, when the devices arecoupled to the network. According to one embodiment of the invention,the communications module 224 may communicate with the servers 120 toexchange data, wherein the data exchange may occur with or without userawareness of the data exchange.

According to an alternative embodiment of the invention, communicationsmay be delayed for an amount of time if, for example, one or more clientcomputers 101, the server 120, the imaging devices 210, and/or thedatabases are not coupled to the network. According to one embodiment ofthe invention, any requests that are submitted while devices are notcoupled to the network may be stored and propagated from/to the offlineclient computer 101, the databases and/or the imaging devices 210 whenthe target devices are re-coupled to the network. One of ordinary skillin the art will appreciate that communications may be conducted invarious ways and among various devices.

According to one embodiment of the invention, user authenticationinformation and/or identification information may be forwarded to theservers 120 to perform various functions. According to anotherembodiment of the invention, the servers 120 may operate to coordinatecommunications between the applications that are associated with theclient computers 101, the imaging devices 210, and/or the databases.

According to one embodiment of the invention, the client computers 101may include, or be modified to include, client computer agent modules226 a-22bn (hereinafter client computer agent modules 226). The clientcomputer agent modules 226 may operate to provide data gathering anddata exchange functionality. According to one embodiment, the inventionmay enable monitoring of actions that are performed on the clientcomputers 101.

According to one embodiment of the invention, the client computer agentmodules 226 may associate client computer identifying information withactions that are performed on the corresponding client computers 101.According to one embodiment of the invention, data monitoring featuresmay be employed to generate client computer audit logs. According to oneembodiment of the invention, client computer audit logs may be producedto reconstruct actions, such as user actions, computer actions, and/orother actions that are performed on (or by) the client- computers 101.

According to one embodiment, the client computer agent modules 226 maygather client computer monitoring data based on user actions performed,such as user login information; data files and databases that areaccessed; information that is requested, including patientnames/identifiers, exam results; information that is retrieved; clientcomputer access information, including user information, time of access,time of exit, etc.; the application(s) that are used; information thatis obtained from the server 120, including time of access, patientidentifiers, volume of data retrieved, etc.; information that isobtained from the imaging devices 210, including time of access, patientidentifiers, volume of data retrieved, etc.; information that isprocessed at the client computer 101, including time stamp information;and other user action data. According to another embodiment of theinvention, user action data may include accessing digital images,reviewing digital images, manipulating digital images, marking digitalimages, storing digital images, forwarding digital images, adjustingexposure parameters on digital imaging devices, generating a report,generating a textual report, dictating a report, entering information,conducting continuing medical education (CME) triggered by performingthe medical examination, and/or performing other user actions.

According to one embodiment, the client computer agent modules 226 maygather client computer monitoring data based on computer actionsperformed, such as when data is exchanged; the type of input deviceused; whether reports are printed; when data is saved; an InternetProtocol (IP) address of devices that are communicated with; a locationof data storage/retrieval; etc.; and/or other computer action data.According to one embodiment of the invention, the client computer agentmodules 226 also may gather client computer specification data, such asIP address data, processing speed data, and other client computerspecification data. According to one embodiment of the invention, theclient monitoring data and/or client computer specification data may beprovided in real-time. According to another embodiment of the invention,the client monitoring data and/or client computer specification data maybe employed to calculate user QA metrics.

According to one embodiment of the invention, the server 120 may includea server authentication module 234 that receives authenticationinformation that is entered into a corresponding client computer 101 viathe authentication modules 222. The server authentication module 234 maycompare the identifying information with existing records and operate asa gatekeeper to the QA scorecard system 100. If the user is determinedto be a registered user, the authentication module 234 may attempt toauthenticate the registered user by matching the entered authenticationinformation with access information that exists on the servers 120. Ifthe user is not authenticated, then the user may be invited to resubmitthe requested identifying information or take other action. If the useris authenticated, then the servers 120 may perform other processing. Forexample, the client computers 101 may receive information from theservers 120 and/or from another authenticated client computers.

According to one embodiment of the invention, the imaging devices 210may include any number of different types of imaging devices, such asmagnetic resonance imaging (MRI) devices, computer tomograph (CT)imaging devices, angiograph imaging device, ultrasound imaging devicesor other imaging devices. According to another embodiment of theinvention, the imaging devices 210 may include several components suchas processors, databases 219 a-219 n (hereinafter databases 219), RAM, aUSB interface, a telephone interface, microphones, speakers, a stylus, acomputer mouse, a wide area network interface, local area networkinterfaces, hard disk drives, wireless communication interfaces, akeyboard, a flat touch-screen display, a computer display, and/or othercomponents.

According to one embodiment of the invention, the imaging devices 210may include, or be modified to include, imaging device agent modules 216a-216 n (hereinafter imaging device agent modules 216). The imagingdevice agent modules 216 may operate to provide data gathering and dataexchange functionality. According to one embodiment, the invention mayenable monitoring of actions that are performed on the imaging devices210.

According to one embodiment of the invention, the imaging device agentmodules 216 may associate imaging device identifying information withactions that are performed on the imaging devices 210. According to oneembodiment of the invention, data monitoring features may be employed togenerate imaging device audit logs. According to one embodiment of theinvention, image device audit logs may be produced to reconstructactions, such as user actions, imaging device actions, and other actionsthat are performed on (or by) the imaging devices 210.

According to one embodiment, the imaging device agent modules 216 maygather image device monitoring data based on user actions performed,such as user login information; imaging modalities; parameters that areselected to perform the imaging modalities, including motioninformation, positioning information, exposure information, artifactinformation, collimation information; number of times an imaging exam isperformed; data files and databases that are accessed; information thatis requested, including patient names/identifiers; information that isretrieved; imaging device access information, including userinformation, time of access, time of exit, etc.; information that isstored to the server 120, including time of storage, patientidentifiers, volume of data stored, etc.; information that was obtainedfrom the imaging devices 210, including time of access, patientidentifiers, volume of data stored, etc.; information that was processedat the imaging device 210, including time stamp information; and otheruser action data.

According to one embodiment, the imaging device agent modules 216 maygather imaging device monitoring data based on imaging device actionsperformed, such as when data is exchanged; the type of input deviceused; whether reports are printed; when data was saved; an InternetProtocol (IP) address of devices that were communicated with; a locationof data storage/retrieval; imaging device parameter adjustments; andother imaging device data. According to one embodiment of the invention,the imaging device agent modules 216 also may gather imaging devicespecification data, such as resolution data, IP address data, processingspeed data, and other imaging device specification data. According toone embodiment of the invention, the imaging device monitoring dataand/or imaging device specification data may be stored in database 219.According to one embodiment of the invention, the imaging devicemonitoring data and/or imaging device specification data of the program110 may be provided in real-time. According to another embodiment of theinvention, the imaging device monitoring data and/or imaging devicespecification of the program 110 may be employed to calculate user QAmetrics. The inventor has previously submitted an application describingan apparatus for automating QA in medical imaging, as described in U.S.patent application Ser. No. 11/412,884 filed on Apr. 28, 2006, theentire contents of which are hereby incorporated by reference.

According to one embodiment of the invention, the server 120 may includea merging module 231 that receives data from all devices that arenetworked to the server 120, including the client computers 101, theimaging devices 210, and/or databases. According to one embodiment ofthe invention, the received data may include at least client computeraudit log data and/or image device audit log data. The merging module231 may locally store the received data in a storage device 260 and/ormay store the received data in an external storage device. The mergingmodule 231 merges data that is captured during a medical examination,including user action data, client computer action data, imaging deviceaction data, and other data.

According to one embodiment of the invention, the server 120 may includea sorting module 240 that enables sorting of the data, including themerged data. According to one embodiment of the invention, the sortingmodule 240 may sort the data based on various sorting criteria,including the chronology of data receipt, the type of device thatoriginated the data, the type of technology used to obtain the data(e.g. CT, MRI, sonogram, etc.), the type of institution in which a datawas obtained, the type of professional that obtained the data (i.e.,radiologist, technologist, etc.), the user that is associated with thedata, the patient that is associated with the data, demographicinformation, patient population information, the workflow sequence inwhich the data was captured, the severity of exam results, and/or othersorting criteria. According to one embodiment of the invention, thesorted data may enable tracking, reconstruction, reporting and/ormonitoring of actions that are performed during medical examinations.According to one embodiment of the invention, the criteria associatedwith medical examinations may be used by the program to calculate QAscorecard metrics.

According to one embodiment of the invention, the server 120 may includea communications module 230 that communicates with the client computer101, imaging devices 210 and/or databases to obtain informationregarding the status of the imaging study along a defined workflowsequence. According to one embodiment of the invention, a definedworkflow sequence includes various operations, such as image examordering, image exam scheduling, image exam acquisition, imageprocessing, image archiving, image navigation, image interpretation,image exam reporting, image exam communication, and/or image exambilling. According to one embodiment of the invention, thecommunications module 230 provides the status of the imaging studyworkflow sequence including identifying the current user that isresponsible for the image study, a completion percentage of the currentstage of the imaging study, and/or other status information. Accordingto one embodiment of the invention, the status of the imaging studyworkflow may be communicated to users in real-time and/or stored.According to one embodiment of the invention, parameters may be derivedfrom the status of the imaging study workflow sequence by the program togenerate a QA scorecard for the various users.

According to one embodiment of the invention, the server 120 may includea report generating module 238 that generates reports based on theoccurrence of pre-defined events, including a periodic query of thestatus of the imaging study, an interpretation that is forwarded by theradiologist, a clinical finding that is submitted by the clinician,and/or the occurrence of other pre-defined events.

According to one embodiment of the invention, the server 120 may includea billing module 242. According to one embodiment, the billing module242 performs billing functions following completion of thereporting/communication process. The billing module 242 may analyzemetrics to assess operational efficiency and accuracy of charges billedand to calculate any additional expenses that occur due to limitationsin reporting by users, such as radiologists. According to oneembodiment, the additional expenses may take a number of forms and mayresult from uncertainty and equivocation within the radiology report orradiologist recommendations for additional imaging exams, consultations,and procedures (e.g. biopsy). The billing module 242 may correlateimaging costs with quality of service deliverables, such as diagnosticaccuracy and clinical outcomes.

According to one embodiment of the invention, the server 120 may includea scheduling module 244 that enables electronic scheduling, includingimage exam scheduling. According to one embodiment, the schedulingmodule 244 may include bi-directional electronic scheduling thatprovides real-time tracking features to update parties of schedulingchanges. The scheduling module 244 may communicate with thecommunication module 230 and/or the notification module 236, among othermodules, to communicate the status of an appointment to users inreal-time and/or stored.

According to one embodiment of the invention, the server 120 may includea notification module 236 that generates notifications and/or alertsbased on the completion of reports, scheduling or the occurrence ofpredefined events. The notifications may be triggered by the release ofitems, such as status information, completion of an imaging report,changes to appointments, and/or other items. The notification module 236may include monitoring features and/or confirmation features that trackand record events, including the date and time that a notification issent, the date and time that a notification is delivered, the date andtime that a notification is opened, such as by return of an acknowledgereceipt message, among other events. According to one embodiment, thenotification module 236 may generate and forward notifications and/oralerts to client computers 101 and/or mobile devices, using knowncommunication techniques including electronic mail messages, voicemessages, telephone messages, text messages, instant messages,facsimile, and/or other communication techniques.

According to one embodiment of the invention, variables that aredetermined to have a deficiency during the imaging study process andthat exceed a pre-determined QA standard threshold may trigger thecomputer program 110 to produce a notification and/or alert through thenotification module 236 that may be instantaneously sent to users, viaone or more communications techniques, alerting users as to the specifictype of deficiency and requirement for correction.

According to one embodiment of the invention, a minimal amount of thedata that is processed at the servers 120 may be stored in storagedevice 260 by the program 110. In other words, the servers 120 mayperform data gathering and/or document generating functions and maythereafter purge all or portions of the retrieved data according tospecified criteria. As a result, according to one embodiment, theprogram 110 may minimize security risks associated with exposing anyconfidential medical records to unauthorized parties at the servers 120.According to another embodiment of the invention, the retrieved data maybe stored at the servers 120 by the program 110 for a predeterminedamount of time before being purged. According to yet another embodimentof the invention, public record information, non-confidential retrieveddata, and/or tracking information, such as client computer log filesand/or image device log files may be stored in storage device 260 by theprogram 110.

According to one embodiment of the invention, the metrics module 232 mayreceive objective scores, such as a Likert scale of 1-4, to quantifyuser performance. For example, a score of 1 may be considered“non-diagnostic”. This means little or no clinically useful (diagnostic)information is contained within the image study. Since the availableinformation obtained during the examination of the patient does notanswer the primary clinical question (i.e., indication for the study),then by definition this requires that the imaging exam be repeated forappropriate diagnosis.

A score of 2 may be considered “limited”. This means that theinformation obtained during the image study is less than expected for atypical examination of this type. However, the information containedwithin the image study is sufficient to answer the primary clinicalquestion. A requirement that this exam be repeated is not absolute, butis preferred, in order to garner maximal diagnostic value.

A score of 3 may be considered “diagnostic”. This means that theinformation obtained during the image study is representative of thebroad spectrum of comparable images, allowing for the patient's clinicalstatus and compliance. Both the primary clinical question posed, as wellas ancillary information, can be garnered from the image for appropriatediagnosis.

A score of 4 may be considered “exemplary”. This means that theinformation obtained during the image study and overall image qualityserves as an example that should be emulated as the “ideal” for thatspecific imaging study and patient population.

According to one embodiment of the invention, the data that is collectedduring the imaging study may analyzed by a metrics module 232 forperforming prospective and retrospective trending analysis. Theprospective and retrospective trending analysis enables automaticdetection of immediate and recurrent problems, as they relate toequipment, personnel, data input, and overall workflow. The result ofthis automated technical QA analysis is that an automated and normalizedanalysis may be performed that minimizes subjectivity and human bias,among providing other benefits.

According to one embodiment of the invention, the metrics module 232 mayautomatically tally and record QA scores in a selected database. The QAscores may be cross-referenced by the computer program 110 to a numberof independent variables including a technologist identifier, imagingmodality, exam type, patient demographics, patient characteristics,patient body habitus, exposure parameters, image processing, examlocation, equipment, day/time of exam for trending analysis, radiologistidentification, referring clinician, clinical indication, among othervariables. According to one embodiment of the invention, the reportgenerating module 238 may access the QA scores to display results fromthe metrics module 232. The reports may be accesses at any time byusers, including the clinician, the radiologist, the technologist,and/or the department/hospital administrator to review individual andcollective performance results. The trending analysis provided by thisdata can in turn be used for educational purposes, performance review,and new technology deployment.

According to one embodiment, the metrics module 232 analyzes data thatis associated with a defined list of quality assurance (QA) benchmarksto objectively evaluate clinicians, quantify a relative success ofservice delivery and provide educational (data-driven) feedback in orderto optimize clinical performance, among other benefits. The QA metricsmay be tied to economic incentives, such as a pay for performance (P4P)systems, to create financial rewards for those practitioners thatprovide high levels of quality-oriented service deliverables.

According to one embodiment, a quantifiable list of pre-defined clinicalperformance parameters may be used by the program 110 to measure overallperformance of the clinician, or practicing physician, such as theutilization and medical imaging services that are provided in a clinicalpractice, among other pre-defined parameters. According to oneembodiment of the invention, clinical performance metrics may becalculated by the program 110 from various parameters, includingcompleteness of data input, such as clinical history, laboratory data,physical exam findings; exam appropriateness, such as using definedappropriateness criteria; utilization patterns, including economicoutcomes, clinical outcomes, and/or medico-legal outcomes; a patientsafety profile, such as requested use of ionizing radiation, contrast,invasive procedures; communication/reporting, including the availabilityof imaging data, the receipt of imaging data, and/or radiologistconsultations; timeliness, including time to initiate clinical action;feedback provided to the patient and specialists, such as theradiologist; participation in data collection and analysis, includingoutcomes analysis, reporting, and/or diagnostic accuracy; education andtraining, including imaging services and new technologies; peer review,including discretionary assessment of clinical performance as it relatesto imaging services and patient diagnosis/treatment, among otherpredetermined parameters.

According to one embodiment of the invention, communication andreporting parameters may include, time from order entry to reportcompletion; time from report completion to receipt by clinician; timefrom report receipt to actual review; specific components of the reportreviewed by clinician; clinician time reviewing reporting data, such asdocument report open and report closing; clinician time components forindividual report segments; perceived clinician value for report; reportstructure; report content; report organization; imaging links, includingcomplete imaging file, key images, snapshot; ancillary data, includingteaching files, NLM, review articles; communication; method ofcommunication; acknowledgement of receipt of communication;bi-directional consultation; time to initiate treatment; tracking offollow-up recommendations; clinician satisfaction; subjective value;referral patterns; among other parameters.

According to one embodiment of the invention, the communication module230 may access a number of informational sources, including theelectronic medical record (EMR); the computerized physician order entrysystem (CPOE); the hospital information systems (HIS) 10, the radiologyinformation systems 20 (RIS); the picture archival and communicationsystem (PACS) 30; subjective feedback from the radiologist, patient, andclinician peer group; and/or other informational sources, to obtainclinical performance parameters. According to one embodiment, standardtags may be created within the various informational sources to identifyindividual QA data parameters.

According to one embodiment of the invention illustrated in FIG. 3, theQA scorecard program 110 presents a welcome screen in operation, orstep, 301. In operation 302, the QA scorecard program 110 displays alog-in screen and receives log-in criteria, such as a username andpassword. In operation 304, the QA scorecard program 110 compares theuser log-in criteria against pre-stored log-in criteria for authorizedusers to determine if the user may gain access to the system. If theuser log-in criteria is not approved, then the QA scorecard program 110may notify the user of the registration failure and may return to themain log-in screen. If the user log-in criteria is approved, then inoperation 306, the QA scorecard program 110 may determine whether or notthe user is assigned full privileges to perform actions within the QAscorecard program 110. If the user has full privileges, then the QAscorecard program 110 requests patient information or a patientidentification number in operation 314.

According to one embodiment of the invention, a reduction of privilegesmay be prescribed for various predefined reasons, including failure tofollow a predefined protocol, frequently misdiagnosing an ailment,failure to follow a cost effective treatment plan, and/or failure tocomplete continuing medical education (CME) credits, among otherpredefined reasons.

According to one embodiment of the invention, if the user does not havefull privileges, then in operation 309, the QA scorecard program 110displays a warning message on the user interface advising the user thatless than full privileges are associated with the log-in criteria. Inoperation 310 the QA scorecard program 110 may identify and recommendre-credentialing programs, including approved CME courses, computertraining, or other re-credentialing programs. The QA scorecard program110 will enable the user to immediately access the recommendedre-credentialing programs through the QA scorecard program 110.According to one embodiment, the user may defer starting the recommendedre-credentialing programs until a future date. After users successfullycomplete the re-credentialing program, the QA scorecard program 110 mayrestore full privileges to the user.

According to one embodiment of the invention, policies governing removaland re-institution of imaging privileges may be under the jurisdictionof a multi-disciplinary QA team including radiologists, administrators,and chief technologists, who would all have input into the overallprocess of reviewing data from the CPOE system.

According to one embodiment of the invention, the warning messagedisplayed in operation 309 also may identify a grace period that isgranted for regaining full privileges before all the privileges arerevoked. According to one embodiment of the invention, the grace periodmay be defined by a threshold, such as a number of log-ins, a number ofdays, or other threshold that may not be exceeded before all privilegesare revoked.

According to one embodiment of the invention, the grace period thresholdmay be determined by the program 110 based on predetermined factors,such as the frequency of occurrence of one of predefined triggers, theseverity of a clinician error, the amount of time required to complete are-credentialing program, and/or other factors. In operation 312, the QAscorecard program 110 determines if the user has exceeded the allowednumber of grace period log-ins. If the number of allowed grace periodlog-ins are exceeded, then the QA scorecard program 110 may revoke allprivileges and the user may be presented with an alert that allprivileges are revoked. The QA scorecard program 110 may prevent theuser from proceeding further in the QA scorecard program 110 and the QAscorecard program 110 may return to the welcome screen. The QA scorecardprogram 110 may provide the user with contact information forre-establishing privileges.

If the grace period threshold has not been exceeded, then the QAscorecard program 110 prompts the user for patient information or apatient identification number in operation 314.

In operation 316, the QA scorecard program 110 accesses one or moreinformation sources, including the electronic medical record (EMR), thehospital information system 10 (HIS), the radiology information system20 (RIS), the PACS 30, among other information sources to obtaininformation and/or records associated with the selected patient.

In operation 318, the QA scorecard program 110 displays the informationand/or records that are associated with the selected patient. Forexample, the QA scorecard program 110 may display an imaging data sheetthat is customized by a user for the patient. According to oneembodiment of the invention, the imaging data sheet provides users withimportant aspects of the patients medical history. The imaging datasheet may have a standard format and include data, such as past medicaland surgical history; prior imaging exams and results, including thoseperformed at outside facilities; current clinical problems; pertinentfindings on physical exam; pertinent laboratory and/or pathology data;ancillary data, including procedural findings (e.g. colonoscopy,bronchoscopy), operative or consultation notes, clinical testing (e.g.,EEG, EKG); technical information related to the imaging exam performed;technologist observations, including pertinent findings andmeasurements; technologist notes, including complications, examlimitations; among other data. The QA scorecard program I 1O facilitatescreation of a universal, patient-specific imaging datasheet for digitalimages that could be stored in the EMR, RIS, and/or PACS, among otherinformation systems.

According to one embodiment of the invention, each time a new entry ormodification is made to the imaging data sheet, a time-stamp may beincluded in the record by the program 110, along with the identificationof the person inputting (or modifying) the data. According to oneembodiment of the invention, each user may create profiles for theimaging data sheet and may customize the imaging data sheet display totheir own individual preferences. According to one embodiment of theinvention, the customized imaging data sheet may be linked by theprogram 110 to users via a log-in criteria.

According to one embodiment of the invention, new data may be input intothe imaging data sheet via the QA scorecard program I 0 by clinicians,nurses, radiologist, technologist or other authorized users. Accordingto one embodiment of the invention, new data may be input into theimaging data sheet via the QA scorecard program 110 throughcomputer-derived entry using natural language processing (NLP).According to one embodiment of the invention, the imaging data sheet mayhave separate tabs for each individual imaging modality, and may storetechnical data, measurements, and technologist notes specific to eachindividual exam/modality.

In operation 320, the QA scorecard program 110 presents the clinicianwith a computerized physician order entry (CPOE) application to initiatean exam order. In operation 322, the QA scorecard program 110 receivesdata that includes all pertinent clinical data elements that are relatedto the diagnosis being evaluated. According to one embodiment of theinvention, the data elements include past medical and surgical history;allergies, with particular emphasis directed to contrast media used inmedical imaging; risk factors, including family history and tumormarkers; non-imaging data, including laboratory, clinical testing,pathology; clinical indication and presumptive diagnosis, which promptedthe ordered imaging exam; findings on physical examination; historicalimaging data, including outside imaging exams and findings; and/or otherdata elements.

According to one embodiment of the invention, a standard tag may becreated by the program 110 within the various informational sources toidentify individual QA data parameters. The communication module 230 mayextract the parameters from the CPOE entries to calculate metrics andgenerate a QA score for the clinician. For example, the metric module232 may reduce a QA score if data elements are “missing,” such as if keyinformation fields are not filled in or are incomplete. In this case ofmissing elements, the QA scorecard program 110 may not process therequest.

According to one embodiment of the invention, the radiologist and/ortechnologist may review the data elements that are entered into the CPOEsystem before the requested imaging exam is performed. The availabilityof the data elements provides an opportunity for the technologist and/orradiologist to clarify any discrepancies or clinical questions. The QAscorecard program 110 enables the technologist and/or radiologist tomake adjustment to the exam protocol and/or to optimize the image examprior to performing the image exam. According to one embodiment of theinvention, discrepancies are defined to include data that isinconsistent with other information that is included in the record. Forexample, the clinician may input data indicating that no prior historyof cancer exists. However, a prior imaging report may show past medicalhistory of cancer.

In operation 324, the QA scorecard program 110 determines whether or notthe clinician's image exam order is appropriate in view of clinicalimaging variables. Exam order appropriateness is a quantitative analysisthat evaluates the clinical efficacy of the image exam ordered, based onan evaluation of the data elements associated with the examinationrequest. According to one embodiment of the invention, the QA scorecardprogram 110 may objectively track the exam order appropriateness usingpre-defined appropriateness criteria, such as linking clinical andhistorical data with different types of imaging exams. For example, if aclinician orders a chest CT to evaluate for lung cancer without firstordering a chest radiograph, the CPOE system may require the lessexpensive screening study (radiograph) to be performed before performinga more expensive CT. If, for example, the patient is being evaluated forkidney stones (urolithiasis) and has a past history of allergic reactionto intravenous contrast dye, the CPOE system will recommend anon-contrast CT or ultrasound (US) in lieu of an intravenous pyelogram(IVP), which requires contrast.

If the exam order is determined to be inappropriate, then the QAscorecard program 110 may display recommendations in operation 325 formodifying the imaging exam order. According to one embodiment of theinvention, the QA scorecard program 110 may use algorithms to generatemandatory and optional recommendations for the clinician. For example,if the patient has a past history of allergic reaction to contrast, thenan IVP is contraindicated and cannot be performed. If, on the otherhand, both a CT and US will provide comparable data, the program 110 mayrecommend the US over the CT, at least due to the fact that US does nothave ionizing radiation, while CT does. In the end, the QA scorecardprogram 110 defers to the discretion of the ordering clinician.According to one embodiment of the invention, the willingness andavailability of the requesting clinician to receive and modify the examrequest may be included by the program 110 in the exam appropriatenessanalysis.

According to one embodiment of the invention, availability may definedas the ability to communicate in a timely fashion (i.e. accessibility).For example, availability may be a combined measure of the timerequirements to document receipt of data and confirm a response.Electronic communication pathways may be created by the program 110 toautomate the communication process as defined through each clinicianuser profile. For example, clinicians may prefer text messaging, e-mailalerts, cell phone, faxing, and/or other communication methods.

According to one embodiment of the invention, willingness may be definedas a degree with which an individual modified the imaging requisitionsin accordance with appropriateness criteria data and recommendations ofdepartmental staff. While there are situations where the clinician mayinsist on following a prescribed order, trending analysis may beperformed by the program 110 to demonstrate outliers, in terms of thoseclinicians that consistently over-ride standard guidelines.

The appropriateness criteria are designed to take into account amultitude of clinical and imaging variables and provide objectivefeedback data by the program 110 to the ordering clinician in order tomaximize patient safety, cost, and diagnostic accuracy. According to oneembodiment of the invention, the metrics module 232 may generate a QAscore for the clinician based on an evaluation of the appropriatenessdata.

According to one embodiment, the QA scorecard program 110 may requestthe clinician to provide additional data in operation 325 for furtherevaluating the image exam order. According to one embodiment of theinvention, the QA scorecard program 110 may electronically track, store,and analyze recommendations reading exam appropriateness to createphysician profiles on ordering habits, completeness of input data, andcompliance with appropriateness standards, among other elements. Thisprofile data may in turn be used for clinician education and training.

According to one embodiment of the invention, another component of examappropriateness may be the routine ordering of preventative screeningimaging exams (e.g. screening mammography), in accordance with communitystandards and patient/family risk factors. As genetic markers become anintegral part of the patient clinical profile, these preventativescreening studies will take on greater importance in disease preventionand will also become an important component in the assessment of examappropriateness.

Upon completion of operation 325, the QA scorecard program 110 returnsto operation 322 in order to allow modification of the pertinentclinical data elements. In operation 324, the QA scorecard program 110again evaluates the modified CPOE to determine whether or not theclinician's exam order is appropriate in view of clinical imagingvariables. If the exam order again determined to be inappropriate, thenthe QA scorecard program 110 proceeds to operation 325.

On the other hand, if the exam order is determined to be appropriate,then the QA scorecard program 110 proceeds to operation 326, where thedata derived from this appropriateness analysis may be stored in adatabase, such as RIS 20, among other databases. In operation 328, theQA scorecard program 110 may analyze the patients data obtained from theHIS 10, RIS 20, EMR or other information source against a patient'sclinician profile. In operation 330, the QA scorecard program 110 maypresent an automated list of preventative screening imaging exams forthe selected patient based on surveillance guidelines.

In operation 332, the QA scorecard program 110 may present additionalpreventative screening imaging exams that may be added to the imageorder exam. In operation 334, the QA scorecard program 110 may forwardthe imaging exam to the clinician's staff for scheduling.

According to one embodiment of the invention, the QA scorecard program110 may evaluate the utilization patterns of ordering clinicians duringthe appropriateness evaluation. In operation 334, the QA scorecardprogram 110 may retrieve, store, and analyze the utilization data fromthe HIS 10 and/or RIS 20 and may correlate the utilization data witheach individual patient's clinical profile. According to one embodiment,the correlation may be defined by disease-specific current proceduralterminology (CPT) codes. These codes are contained within the HIS 10 forinpatient hospitalizations and the EMR, among other databases.

According to one embodiment of the invention, the QA scorecard program110 may periodically track and analyze this data in order to provideinsight as to specific clinical indications and diagnoses requiringremedial education on the part of the clinician. The frequency ofanalysis may be established by each individual site and may be performedmonthly or quarterly basis, for example. The QA scorecard program 110may analyze variables, such as patient safety, cost, redundancy, andclinical efficacy, among other variables. According to one embodiment ofthe invention, patient safety may include elements such as contrastadministration, radiation exposure, and invasive procedures (e.g.biopsies). According to one embodiment of the invention, cost mayinclude an analysis that takes into account whether less expensiveimaging studies are being utilized in lieu of more expensive, technologyintensive exams. For example, if a patient presents with headaches, CTis a less expensive exam that provides comparable diagnostic informationto a more expensive MRI. According to one embodiment of the invention,redundancy is the duplication of imaging exams to provide similarimaging data. One of the problems with over-utilization is thatphysicians often order multiple imaging exams that individually answerthe same clinical questions. If, for example, a patient is beingevaluated for elevated liver enzymes there is little yield in orderingboth an abdominal US and CT, yet it occurs quite frequently. Invasiveprocedures are fraught with potential morbidity and mortality. As aresult, they should only be performed after all non-invasive diagnosticwork-ups have been exhausted.

Adverse effects of over-utilization of medical imaging services may bedetermined from these variables. Adverse effects of over-utilization ofmedical imaging services may include economic factors, such as increasedcosts for overall healthcare delivery; timeliness, such as potentialdelay in diagnosis and treatment planning; decreased accessibility, suchas diminished capacity and increased exam backlog; diffusion ofresponsibility, such as increased number of consultants and tests withdiminished clinical focus on primary care provider; increased relianceon technology, such as depersonalization of medical practice; patientsafety, such as increased risk of adverse actions associated withcontrast and ionizing radiation; diminished R & D, such as potential todecrease innovation and new product development due to medico-legalrisk; among other factors.

In operation 336, the QA scorecard program 10 may perform trend analysisof clinician medical imaging utilization. The trend analysis may beevaluated on an individual patient and patient group basis, with patientgroups classified according to demographics, medical histories, andclinical profiles. According to one embodiment of the invention, the QAscorecard program 11O may perform trend analysis to identify specifictrends in imaging utilization patterns. Since patient, institution, andclinical indication are unique, they should be taken in the overallcontext of multiple data points. For example, if a physicianinappropriately orders the wrong study on a single patient than it isrecorded and taken into context. If, on the other hand, that samephysician repeatedly orders the wrong imaging study on multiplepatients, then the overall trend is one which identified a need forintervention, such as in the form of education.

In operation 338, the QA scorecard program 10 may correlate utilizationdata trends with local, regional, and national norms. The QA scorecardprogram 110 may provide data-driven feedback to each clinician relativeto their peer group for educational purposes. The QA scorecard program11O may separate utilization patterns into categories including,preventative medicine, diagnosis, treatment, and disease surveillance,among other categories. In operation 340, the QA scorecard program 110may present feedback to the clinician regarding the utilization data.

In operation 342, the QA scorecard program 10 may derive apply theutilization data to create best practice guidelines and assist intechnology allocation, development of new imaging services, and improvedappropriation of healthcare funding.

Best practice guidelines refer to what are shown to be the mosteffective imaging studies for specific clinical indications, based on alarge number of data from multiple institutions. For example, in theevaluation of osteomyelitis for the diabetic foot, it may bedemonstrated through scientific study, that contrast enhanced MRI is thebest imaging study (analyzing cost, safety, and clinical efficacy), asopposed to alternative imaging exams (CT, radionuclide bone scan orwhite blood cell scan).

Regarding technology allocation, if the QA scorecard program 110generates cumulative data from the utilization analysis showing a needfor a second MRI scanner, than this evidence may be provided to supporta request for a second MRI during the next capital equipment cycle. Inthis regard, some states require certificates of need (CON) for certaintypes of advanced technologies (e.g. PET scanner) and this data providesan objective means to justify (or refute) requests for CON.

Regarding education, the QA scorecard program 110 may distinguishbetween intra-departmental or on-line educational courses that aredesigned to educate clinicians as to new imaging technologies andapplications. The QA scorecard program 110 may also direct users toinformation regarding the best utilization of these services. If forexample, a clinician continues to inappropriately order the same type ofimaging exam for a specific clinical indication/diagnosis, then they maybe made aware of existing recommendations. These educational programsmay be automated and customized to a specific problem by the program110, such as by bookmarking certain on-line education materials tospecific imaging data. For example, maybe a website has an overview onnew MRI applications for neuro-imaging. If a clinician isinappropriately ordering brain MRI exams, the program 110 may identifythe error and direct the clinician to the on-line educational programthat best fits the area of concern. Once the educational program hasbeen completed, as documented by CME credits, for example, the QAscorecard program 110 may restore any temporarily removed privileges.

FIG. 4 illustrates a communication and reporting process that may bepresented to a clinician after a radiologist has interpreted the imagingstudy, according to one embodiment of the invention. Communication andreporting procedures of the program 110 ensure that the informationcontained within the medical imaging report is received in a timelyfashion and appropriately utilized for clinical management. The QAscorecard program 110 may alert the clinician after the radiologist hasinterpreted the images.

In operation 400, the QA scorecard program 110 presents imaging resultsto the clinician from information sources, such as HIS 10, RIS 20, PACS30, among other information sources. In operation 402, the QA scorecardprogram 110 receives an assessment regarding whether or not the imagingstudy contains unexpected results or emergent findings.

If the imaging study does not include unexpected results or emergentfindings, then the clinician is presented with the results by theprogram 110 pursuant to standard protocol. If the imaging study includesunexpected results or emergent findings, then the clinician is presentedwith the results by the program 110 pursuant to special protocol. Whenthe QA scorecard program 110 receives information of clinicallyunexpected or emergent findings by the radiologist during the course ofimaging exam interpretation, the QA scorecard program 110 generates analert that notifies the ordering clinician to immediately contact theradiologist to directly receive these emergent findings.

When the imaging study includes unexpected results or emergent findings,the QA scorecard program 110 sends an alert to the clinician inoperation 404 with instructions for contacting the radiologist. Whetherthe communication occurs electronically or verbally, the QA scorecardprogram 110 documents the communication by time stamping and storing thecommunication for future analysis. Upon receipt of the alert, theclinician may immediately contact the imaging department staff (i.e.radiologists, technologists, administrators) to discuss the clinicalconcerns. In operation 406, the clinician may contact the patient toadvise of the results. The QA scorecard program 110 documents thecommunication between the clinician and patient by time stamping andstoring the communication for future analysis. According to oneembodiment of the invention, QA scorecard parameters for reporting andcommunication may include criteria such as, time from order entry toreport completion; time from report completion to receipt by clinician;time from report receipt to actual review; specific components of thereport reviewed by clinician; clinician time reviewing reporting data,such as document report open and report closing; clinician timecomponents for individual report segments; perceived clinician value forreport; report structure; report content; report organization; imaginglinks, including complete imaging file, key images, snapshot; ancillarydata, including teaching files, NLM, review articles; communication;method of communication; acknowledgement of receipt of communication;bi-directional consultation; time to initiate treatment; tracking offollow-up recommendations; clinician satisfaction; subjective value;referral patterns; among other criteria.

According to one embodiment, the time-stamped data is a component partof objective data analysis. Imaging departments are able to utilizeprogram 110 to record individual time-stamped data throughout the courseof the imaging cycle, from the time an imaging exam is electronicallyordered to the time the imaging report issued and reviewed. After theimage report is received, time-stamped data may be tracked by theprogram 110 within the EMR, which records clinician actions, in the formof recording progress notes, consultations, and the ordering of clinicaltests, imaging studies, and various treatment options (e.g. drugtherapy). In either case, the QA scorecard program 110 enables theclinician to enter data electronically into the EMR. This istime-stamped data may be recorded into a QA database for subsequentanalysis. One such analysis may include an assessment of the timeincurred between the imaging exam and initiation of clinical treatment.

According to one embodiment, a clinician may order a chest CTangiography (CTA) in the assessment of suspected pulmonary embolism. Dueto the emergent nature of the clinical indication, the QA scorecardprogram 110 may be accessed to order the imaging exam (within the CPOEsystem) as “stat”. The exam order time is recorded by the QA scorecardprogram 110, for example at 18:08 hours. The patient arrival time to theimaging department may also be recorded by the QA scorecard program 110,for example at 18:32 hours. The individual components of the examinationperformance time also may be recorded within the RIS by the QA scorecardprogram 110, including the exam begin time, for example at 18:40 hoursand exam completion time, for example at 18:45 hours. The image exam maybe transferred and saved to the PACS upon completion by the QA scorecardprogram 110. Once the exam is saved to the PACS, the QA scorecardprogram 110 may make the exam available to the radiologist and theradiologist may be alerted accordingly. The time that the image exam isrecorded within the PACS, along with the time the imaging exam wasdisplayed and the time dictation was completed may be recorded by the QAscorecard program 110, for example at 19:01 hours. If, in this example,the radiologist used speech recognition software to transcribe thedictated report, the report completion time may be identical to the timedictation was completed, for example at 19:01 hours.

Due to the emergent nature of the imaging exam (ordered stat), the QAscorecard program 110 may immediately send the imaging report to thereferring clinician, such as via a Critical Results Reporting programwithin the PACS. Receipt by the clinician may be acknowledged andconfirmed electronically by the QA scorecard program 110, via theclinician's PDA, for example at 19:10 hours. Based on the positivefindings of pulmonary embolism (on the CTA report), the clinician inturn may access the QA scorecard program 110 to immediately orderinitiation of anti-coagulation therapy, with the time-stamped orderrecorded in the EMR, for example at 19:14 hours.

Since all these events are recorded electronically within the variousinformation systems, they are available to be recorded into the QAdatabase, along with any corresponding analysis. The QA scorecardprogram 110 may calculate metrics from various parameters, including theexam completion time, such as the time from order entry to examcompletion: 37 minutes; reporting time, such as the time from examreviewed by radiologist to time report received by clinician: 25minutes; the time to initiate clinical action, such as the time fromreport receipt to order entry into EMR: 4 minutes; and the total exam totreatment cycle time, such as time from exam order to treatment order:66 minutes; among other parameters.

According to one embodiment, the QA database and time-stamped dataelements within the QA database enable the various time requirements tobe quantified by the program 110 for various components within theoverall imaging/treatment cycle. By doing so, an objective methodologyis provided by the program 110 to assess clinician availability andresponsiveness with regard to imaging examination data. Thistime-stamped data becomes a valuable tool for users, such asadministrators and clinicians, to assess overall workflow and identifybottlenecks and limitations within the system. The QA database alsoprovides valuable data to the community at large as to the timeefficiency of imaging service and clinical providers.

When the imaging study does not include unexpected results or emergentfindings, the process proceeds to operation 408, where the QA scorecardprogram 110 presents the imaging results to the clinician. In operation410, the QA scorecard program 110 may present the clinician with supporttools and facilitate a consultation with the radiologist, as needed. TheQA scorecard program 110 may present decision support tools, includingcomputer-aided detection (CAD), specialized image processing, electronicteaching files and other on-line educational resources for patientmanagement, such as the National Library of Medicine (NLM) forliterature searches.

In operation 412, the QA scorecard program 110 presents the imagingstudy data including exposure levels and other parameters of the imagingstudy. The imaging study data may be provided by the program 110 frommultiple sources, including the imaging modality, such as acquisitionparameters used to calculate radiation dose; contrast injectortechnology, such as contrast-related data; EMR, such as patienthistorical data; and radiology personnel (radiologist, technologist,nurse). In operation 414, the QA scorecard program 110 may store theimaging study data in a storage device associated with RIS 20, amongother storage devices. The imaging study data may be associated by theprogram 110 with clinical feedback information.

In operation 416, the QA scorecard program 110 calculates an amount ofpatient radiation exposure. For example, the QA scorecard program 110may calculate the ionizing radiation that is associated with eachindividual medical imaging exam, based on acquisition and exposureparameters. The calculation may be performed prospectively by theprogram 110 and may be stored to track longitudinal radiation dose andcarcinogenesis risk. In operation 418, HIS 10 and RIS 20, and/or otherinformation sources, may be updated by the program 110 to include thecalculated safety factors.

According to one embodiment, in order to optimize safety concerns andrecord/track cumulative data, the QA scorecard program 110 providespatient safety data at any location where the patient is seeking and/orreceiving medical imaging services. In the event that the patient had aprevious allergic reaction to contrast at another medical imagingfacility and the patient now presents with altered mental status andcannot provide accurate historical data, the QA scorecard program 110provides access to the pre-existing safety data before any imaging examis performed. By storing the QA Scorecard data within a universal EMR,this data is made accessible by the program 110 to appropriatehealthcare providers at any location.

One method of providing universal accessibility for the data is to useextensible mark-up language (XML). XML further enables communicationbetween disparate information technologies by allowing creation of astandard tag for individual QA data parameters. According to oneembodiment, QA metrics may be employed to define XML tags, such asexamination time, technologist retake, reject analysis, among other QAmetrics. According to one embodiment, XML tags may be communicated amonginformation technologies, such as modalities, information systems, PACS,EMR, CPOE. According to one embodiment, XML tags may be automaticallydownloaded into a universal QA database.

According to one embodiment, the QA scorecard program 110 may track,record and analyze longitudinal patient-specific safety data andclinician-specific safety data, both an individual patient and groupbasis. This provides insight as to whether individual clinicians areover-utilizing certain types of “higher risk” imaging studies andprovides educational feedback to specific clinicians. Additionally,mandatory educational resources may be forwarded to targeted cliniciansfor completion before imaging privileges are re-instated. This“clinician safety profile” data and trending analyses may be correlatedby the program 110 with local, regional, and national norms, with dataavailable to third party payers and insurers to assist with economicincentive programs (P4P) to encourage improved performance andcontinuing medical education, as it relates to medical imaging safetyfactors.

In operation 420, the QA scorecard program 10 may determine whether ornot patient safety factors are exceeded. If patient safety factors areexceeded, then the QA scorecard program 110 may refer the radiologistand technologist to a QA committee in operation 422.

If patient safety factors are not exceeded, then the QA scorecardprogram 110 may proceed to present the clinician with external peerreview results in operation 424. External peer review serves as amechanism for each clinician to be evaluated both prospectively andretrospectively by their medical peers. According to one embodiment,prospective evaluation may be provided by radiologists that serve asimaging consultants and provide feedback as to the efficacy of imagingexam utilization. According to one embodiment, clinician peers mayprovide both prospective and retrospective feedback as to the efficacyof clinical management, following completion of the initial imagingexam. According to one embodiment, utilization of consultants,laboratory and clinical tests, and invasive procedures (e.g. surgicalbiopsy) play an important role in diagnosis, while medical, surgical,and radiation therapy all play a role in disease treatment. By randomelectronic auditing of the CPOE and EMR, peer review can be directlyincorporated into patient management and provide an important subjectivetool for providing feedback to the clinician on imaging service andclinical management. The data derived from this peer review would beentered into a comprehensive QA database and provided to the clinicianon a periodic basis for educational purposes, with trending analysesdocumenting “best practice” guidelines relative to local, regional, andnational peer groups.

According to one embodiment, combined subjective and objective feedbackprovides data to clinicians as to how their performance is perceived byimaging service providers, such as radiologists, and customers, such aspatients. Radiologist may provide feedback regarding availability andaccessibility for reporting, consultations, and queries related to thepatient and exam ordered. In addition, radiologists may provide data asto the frequency and reliability of clinical feedback provided to themby the clinicians. For example, if a radiologist reported a nonspecificlung nodule on a chest CT exam and recommended PET scan correlation, itis instructive for that radiologist to receive follow-up information, inthe event that the recommended PET scan was performed at an outsideinstitution. According to one embodiment of the invention, feedback maybe provided in an automated fashion by the program 110 with theintroduction of pop-up menus and electronic auditing tools that trackclinician review of the imaging report and record their preferences. Thebi-directional nature of this feedback is ultimately aimed at improvingclinical outcomes and is equally important to both the radiologist andthe clinician. Clinician feedback is critical in the overall evaluationof medical imaging services, including report accuracy, recommendedfollow-up, and timeliness in diagnosis.

According to one embodiment of the invention, the clinician may alsoobtain feedback from the patient via electronic and/or paper surveys.The feedback may include the patient's subjective perceptions as to anumber of factors, including conscientiousness, education,responsiveness to questions, communication skills and/or timeliness,among other factors. The feedback may be obtained from the patientbefore the patient leaves the imaging department or later via electronicor conventional mail. Questions may be posed using a Likert scale toprovide a quantitative value. Because answers tend to be biased (somepeople grade too harshly, others too easily), the scored answers may beextrapolated by the program 110 based on individual biases relative tothe larger group. Subjective answers may be reviewed by an impartialadministrator who records information into the QA database to identifyconsistent trends.

Active participation in prospective data collection and analysis is acomponent of the QA Scorecard and pay for performance system. The datathat is collected and analyzed by the program 110 provides thefoundation for evaluating reports, diagnostic accuracy, and clinicaloutcomes analysis.

It should be emphasized that the above-described embodiments of theinvention are merely possible examples of implementations set forth fora clear understanding of the principles of the invention. Variations andmodifications may be made to the above-described embodiments of theinvention without departing from the spirit and principles of theinvention. All such modifications and variations are intended to beincluded herein within the scope of the invention and protected by thefollowing claims.

1. A quality assurance system for radiology, comprising: at least oneclient computer; at least one imaging device, comprising: an agent thatcaptures user action data based on actions that are performed on theimaging device; a server comprising: a communications module thatcommunicates with the agent to receive the user action data and the atleast one client computer; and a notification module that monitors theuser action data and forwards an alert to the client computer based onan occurrence of a predefined event.